1. Technical Field
The present invention relates to haptic augmented reality, and more particularly, to a method for providing haptic augmented reality that provides mixed results of a real environment and a virtual environment to a user.
2. Related Art
Imagine that you are holding a pen-shaped tool in your hand and writing something on a table. Would it be possible that you may feel as though you were writing on a smooth piece of paper with a ball-point pen, or on a soft rubber pad with a marker? The tool may also guide your hand to teach the art of East Asian calligraphy, preferably with the feel of using a brush on a piece of traditional East Asian paper. Creating such haptic illusions belongs to the realm of haptic augmented reality.
Augmented Reality (AR), or more generally, Mixed Reality (MR) environment provides the user with sensations resulted from a mix of real and synthetic stimuli that are generated by computer. As conceptualized by Milgram et al., a mixed environment can be located at a reality-virtuality continuum (Milgram, P., Colquhoun Jr., H.: A Taxonomy of Real and Virtual World Display Integration. In: Tamura, Y. (ed.) Mixed Reality. Merging Real and VirtualWorlds, pp. 1.16. Springer, Berlin (1999)) Whether an environment in this continuum is closer to reality or virtuality is determined by the amount of knowledge that the computer is required to manage for the environment. For example, the display with head of a human in an aircraft/automobile cockpit is an application of augmented reality by coating a real photograph of a face image of a celebrity on a virtual body employed in a computer game is classified to be an example of augmented virtuality. Although the continuum was primarily defined for vision, the same continuum can be applied to touch. Given a mixed reality application including both vision and touch, the degrees of reality (or virtuality) for the two sensory modalities can be identified and mapped to the composite visuo-haptic continuum of reality and virtuality.
Earlier research on haptic mixed reality can be categorized using the continuum proposed by Milgram, et al. Although the taxonomy in the continuum is not strictly adhered, even in the literature of visual mixed reality, applying it to haptic mixed reality can be instrumental for elucidating associated concepts. Firstly, “haptic reality” corresponds to applications wherein a user directly touches real objects and feels only a touch from the real object. A typical example is the so-called tangible AR where a real prop held in the hand is used as a tangible interface for visually mixed environments. Secondly, “haptic virtuality” is located at the other end of the continuum, and has received the most attention from the research community. In this case, purely virtual haptic objects are added in a vision mixed reality environment, and the touch of the virtual object is rendered based on the conventional haptic rendering methods under the virtual environment. Thus, accurate positional registration between the haptic and visual coordinate frames is a key issue. The third category is “haptic augmented virtuality”. In this environment, a real prop attached to the haptic device to provide the sensation of surface real material and the haptic device performs only a role of a force-feedback device of the virtual object.